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ABSTRACT 

This  thesis  compared  the  use  of  a  discrete  utterance 
voice  recognition  system,  and  a  keyboard  entry  device  in 
retargeting  Air  Launch  Cruise  Missiles  ( ALCM)  prior  to 
launch  from  a  E-S2G  aircraft.  Time  to  load,  input  and 
output  accuracies,  and  time  versus  accuracy  measurements 
were  trade  for  each  of  twenty  subjects.  Keyboard  entry  was 
found  to  be  tetter  than  voice  entry  in  time  to  lead  and 
input  accuracy.  These  findings  are  limited  to  discrete 
utterance  voice  recognition  systems  and  most  probably  would 
have  been  different  if  a  connected  speech  recognition  system 
had  been  utili2ea. 

Also  investigated  were  three  display  formats  for 
presenting  flex  targeting  information  on  a  cathode  ray  tube. 
Information  was  updated  on  the  cathode  ray  tube  using 
single-space,  double-space,  and  inverse-video  formats.  Time 
to  update,  input  and  output  accuracies,  and  time  versus 
accuracy  measurements  were  recorded.  No  significant 
differences  were  found  among  the  three  display  formats. 
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I.  BACKGROUND 


A.  INTRODUCTION 

The  E-52G  va$  selected.  in  1977  to  be  outfitted  with  an 
all  new  navigation  and  weapons  control  suite  called  the 
Offensive  Avionics  System  (CAS).  This  systerr  would  control 
and  launch  the  Air  Launch  Cruise  Missile  (ALCM).  The  flight 
crews  based  at  the  Eoeicg  Airplane  Company,  Wichita,  Kansas, 
and  the  human  factors  staff  at  Edwards  AFE,  California,  were 
tasked  to  evaluate  the  OAS .  3ased  on  conversations  with 
these  two  groups,  it  was  discovered  that  the  ALCM 
retargeting  procedure,  also  known  as  "flex  targeting”,  was 
slow  and  cumbersome. 

The  purpose  of  flex  targeting  is  to  give  political  and 
military  leaders  greater  flexibility  during  both  strategic 
and  tactical  conflicts.  In  future  warfare,  the  political  as 
well  as  the  military  situation  will  tie  in  constant  flux.  As 
the  situation  rapidly  changes,  targets  will  change  as  well. 
A  commander's  ability  to  change  the  course  of  a  battle  in 
minutes  or  seconds  will  oe  dependent  upon  the  ability  of  his 
forces  to  respond  to  these  sudden  changes.  This  flexibility 
depenis  upon  rapid  weapon  retargeting  which  the  present  OAS 
does  not  provide. 

One  possible  solution  to  this  problem  was  seen  in 
computer  voice  technology.  Work  in  this  area  of  computer 
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vcice  applications  research  Is  ongoing  here  at  the  Naval 
Postgraduate  School  by  Dr.  Gary  Poock.  If  was  felt  that  the 
AICM  flex  targeting  problem  could  be  easily  investigated 
here  due  to  the  extensive  facilities  of  the  Human  Factors 
laboratory. 

B.  OAS/ALCM  SYSTSM  IN  E-52G 

1.  Background 

The  B-52  bomber  has  been  in  the  U.S.  Air  Force 
arsenal  since  1954.  Seven  nundred  forty-four  aircraft  were 
produced  through  1962  beginning  with  model  "a"  and  finishing 
with  model  ’h".  Today  only  three  models,  the  "d",  "g",  and 
"K"  remain  in  service. 

Ail  three  are  assigned  nuclear  and  conventional  roles. 
The  ”D"  model,  the  oldest  and  least  modified  of  the  three, 
is  expected  to  he  retired  within  the  next  two  years  and  he 
replaced  by  103  B-lB  aircraft.  (The  USAF  has  designated  the 
cruise-mlssile-capabie  E-l  the  'B-lB'.  It  will  he  equipped 
with  external  cruise  missile  pylons.) 

The  and  ”h"  models  continue  on  active  duty  and  are 
the  subjects  of  many  modifications  to  extend  their  life, 
improve  their  survivability,  and  add  to  the  missions  they 
are  already  expected  to  perform.  This  paper  addresses  a 
mission  added  recently  to  the  B-52G,  which  is  presently 
undergoing  modifications  which  will  allow  It  to  carry  the 
Air  Launched  Cruise  fissile  lALCM). 
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2.  Operation 


The  ALCM  is  a  spall,  unmanned,  winged  air  vehicle 
capable  of  sustained  subsonic  flight  following  launch  from 
the  B-52G  (see  Figure  1).  Propelled  by  a  turbofan  engine, 
it  incorporates  a  nuclear  warhead  with  a  variable  yield  of 
up  to  200  kiiotons  for  a  "hard  target"  kill  capability,  and 
is  programmed  tc  strike  a  pre-determined  surface  target 
[?.ef .  1], 

While  the  ALCm  is  onboard  the  B-52G,  it  is  receiving 
position,  heading,  and  altitude  information  from  the 
aircraft's  inertial  navigation  system  every  60  seconds  [P.ef. 
2].  When  the  missile  is  activated  for  launch,  it  knows 
where  it  is  and  where  it  is  heading. 

Upon  launch  from  the  B-62G,  ALCM  wing  deployment  is 
attained  within  two  seconds,  and  engine  start  within  three 
seconds.  After  launch,  position,  attitude  and  velocity 
reference  aata  are  derived  from  the  AICM's  inertial 
navigation  system,  mach  number  and  pressure  altitude  from 
the  air  data  system,  and  height  above  the  terrain  from  the 
radar  altimeter.  Initial  position/velocity  data  and 
atmospheric  paraneters  are  transferred  to  the  missile  from 
the  B-5ZG  carrier  immediately  prior  to  launch.  The  missile 
is  then  guided  inertially  along  preplanned  flight  profiles 
at  programmed  Mach  numbers  and  altitudes.  Velocity  commands 
are  based  on  precomputed  ground  speeds  required  to  reach  the 
target  at  a  specified  tire. 
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Figure  1.  Air  Launch  Cruise  Missile  (ALCM) 


”Tim;e-to-target "  is  calculated  by  the  computer,  and  throttle 
commands  are  issued  as  required  to  maintain  average  ground 
speeds  along  each  trajectory  segment.  Altitudes  are 
selected  to  be  either  "altitude  hold",  where  the  missile 
flies  at  a  commanded  barometric  altitude,  or  "terrain 
following"  where  the  missile  flies  at  a  specified  height 
above  the  terrain. 

Guidance  is  by  a  combination  of  an  inertial  navigation 
system  and  a  terrain  comparison  (TERCOM)  technique  which 
utilizes  the  radar  altimeter  and  onboard  computer.  At 
predetermined  points  along  its  flight  path,  the  missile  uses 
its  radar  altimeter  to  map  the  profile  of  the  terrain  below. 
This  "sensed  profile"  is  compared  with  a  preloaded  digital 
map  of  the  surrounding  area.  (Mapping  information  will  be 
furnished  by  the  Defense  Mapping  Agency.)  Locating  itself 
within  the  map,  the  missile  computer  determines  its  true 
position,  velocity,  and  heading  and  maltes  course  corrections 
if  necessary.  At  first  landfall,  the  missile's  map  may  be 
of  an  area  measuring  in  the  tens  of  miles.  When  it  updates 
its  position,  it  also  adjusts  drift  rates,  calibrations,  and 
other  guidance  error  contributions  to  make  itself  less  and 
less  dependent  upon  map-correlation  updates.  The  radar 
altimeter  then  goes  into  "standby"  until  it  enters  the  next 
position  update  area.  Thus,  as  the  missile  proceeds  toward 
its  target  the  maps  would  be  of  smaller  and  smaller  areas, 
since  the  guidance  system  fine-tunes  out  practically  all 
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error.  TERCOM  is  said  to  resolve  objects  on  tbe  ground  as 
small  as  three  meters  wide  and  30  centimeters  tall  from  an 
altitude  of  several  thousand  meters  [Ref.  3].  The  TERCOM 
feature  allows  the  AICM  to  fly  at  lew  operating  altitudes 
below  100  feet  [Ref.  3].  Maximum  operating  altitude  for  the 
radar  altimeter  is  5000  feet  above  the  ground.  All  terrain 
following  is  performed  below  this  altitude.  In  the  event  of 
radar  altimeter  failure  during  terrain  following,  the 
missile  automatically  transfers  to  the  altitude  hold  mode 
and  flies  at  a  preset  baronetric  altitude  stored  in  the 
mission  software  (Figure  11,  line  #4) .  This  value  is 
selected  to  provide  adequate  terrain  clearance  for  the  area 
being  overflown. 

This  high  accuracy  of  missile  position  updating  results 
in  extreme  accuracies  at  the  target.  U.S.  cruise  missiles 
to  date  have  achieved  accuracies  on  the  order  of  30  feet  CEP 
[Ref.  3],  This  means  at  least  50“  of  the  time  the  missile 
will  land  within  30  feet  from  the  target. 

The  AL CM s  carried  by  the  B-52G  are  programmed  initially 
with  one  target  per  missile.  Anytime  prior  to  ALCM  launch, 
a  new  target  can  be  entered  into  the  missile  by  the 
navigator.  This  retargeting  capability  Is  referred  to  as 
"flex  targeting".  Using  the  current  integrated  keyboard 
<IO),  this  procedure  requires  approximately  1-2  minutes  per 
missile  to  complete.  Retargeting  a  full  load  of  twenty 
missiles  would  require  20-40  minutes. 


Accelerated  production  for  the  USAF/Boeing  ALCM  at 
production  rates  of  480  missiles  per  year  will  start  in 
Fiscal  1984.  This  plan  is  predicated  on  the  deployment  of 
3,780  missiles  by  1990  on  a  schedule  consistent  with  B-32G/H 
conversions  [Ref.  4]. 

3 .  Bole  of  Navigator 

Responsibilities  for  navigation  and  weapons  control 
in  the  B-52G  are  assigned  to  the  navigator  and  radar 
navigator.  The  navigator  determines  aircraft  position  based 
on  radar  information  obtained  by  the  radar  navigator.  He 
compares  actual  position,  course,  etc.,  to  preplanned 
mission  course  and  directs  the  aircraft  accordingly.  The 
navigator  is  also  responsible  for  ALCM  status.  If  a  change 
in  ALCM  targets  is  directed  by  appropriate  authorities,  he 
would  be  the  crewmember  to  make  these  changes. 

Except  during  the  flex  targeting  procedure,  there  are  no 
other  inputs  the  navigator  must  perform  with  regard  to  the 
A ICtv .  He  does  have  a  constant  missile  status  of  each,  and 
if  one  malfunctions  prior  to  launch,  it  can  be  withheld  by  I 

the  navigator  ana  jettisoned  if  necessary.  , 

4.  Scenarios:  Tactical  &  Strategic  ! 

The  cruise  missile  carried  onboard  the  B-52G  will  be 

capable  of  performing  non-nuclear  (tactical)  roles  as  well 
as  its  main  role  in  nuclear  (strategic)  scenarios. 

In  the  tactical  scenario  rapid  retargeting  of  AICMs 
could  give  defending  forces,  especially  in  the  European 
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Theater,  an  unprecedented  capacity  for  disrupting  Soviet 
Pact  supply  lines,  destroying  support  facilities,  and 
attaching  front-line  forces  during  a  rrassive  thrust  across 
Western  Europe.  A  £-b2G  orbiting  outside  the  battle  area 
could  respond  to  battlefield  commanders  by  launching  ALCMs 
against  targets  tcc  risky  for  aircraft  strikes,  cr  toe  far 
away  from  artillary  positions.  Since  tactical  targets  are 
very  mobile,  a  rapid  and  accurate  ALCM  retargeting 
capability  is  essential.  If  a  voice  entry  system 
significantly  improved  speed  and  accuracy  then  considerable 
capability  would  be  added  to  the  flex  targeting  option. 

Utilizing  a  nuclear  warhead  in  a  strategic  role,  cruise 
missiles  can  strengtnen  the  air  leg  of  the  strategic  triad 
of  the  U . S .  which  includes  intercontinental  ballistic 
missiles  (ICBMs),  submarine  launched  ballistic  missiles 
(SLBMs) ,  and  strategic  bombers.  ALCMs  will  give  heavy 
barbers  a  standeff  capability,  and  the  missile's  long  range 
miles)  would  lessen  the  need  for  bomber  refueling 
[Ref.  5].  Planners  could  assign  ALCM-equipped  3-52Gs  a 
standoff  role  of  firing  their  missiles  outside  Russian 
airspace  without  actually  penetrating,  or  firing  and  then 
penetrating  to  drop  their  gravity  bombs,  cr  launching  ALCMs 
within  Russian  airspace  as  they  penetrate.  If  the  flex 
targeting  procedure  near  cr  within  these  high  threat  areas 
was  exercised,  the  1-2  minutes  required  for  retargeting  each 
ALCM  would  increase  the  bomber's  probability  of  detection  by 
enemy  radar. 
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C.  SPEECH  RECOGNITION  &  CRT  DISPLAYS 


1 .  Overview 

Ever  since  machines  have  been  used  by  man,  he  has 
sought  ways  to  have  rrore  control  over  them.  With  the 
development  of  computers,  man  continued  to  exercise 
mechanical  control  by  means  of  keyboard  interfaces  or 
manually  punched  cards.  The  ability  to  directly  communicate 
with  them  ty  voice  was  first  envisioned  by  science  fiction 
writers  of  the  1960's  and  cO's.  As  the  computer  became 
faster  and  able  to  store  larger  amounts  of  information, 
researchers  began  investigating  the  possibility  of  speech 
recognition  ty  computer. 

The  theory  behind  computer  speech  recognition  (also 
Known  as  voice  data  entry)  was  simple  enough,  at  least  for 
isolated  words. 

The  basic  theory  is  as  follows: 

1.  Air  pressure  variations,  created  by  speech,  are  first 

converted  into  an  electrical  signal  by  microphone. 

2.  The  signal's  acoustic  features  are  converted  into 

computer  machine  language,  and  stored  in  computer 
memory . 

2.  Pecause  a  word  does  not  necessarily  have  the  same  length 
each  time  a  speaker  says  it,  the  computer  must  "time 
align"  its  length. 

4.  The  computer  then  compares  the  features  of  the  "aligned" 


won  against  utterances  already  stored  in  its  memory. 
The  computer  decides  whether  the  spoken  word  matches  a 
word  in  memory. 

5.  Assuring  a  natch  is  found,  it  then  performs  the  action 
or  actions  (pre-programmed;  corresponding  to  that 
utterance . 

The  computer  thus  recognizes  an  utterance,  or  word, 
within  a  given  time  frame.  It  also  rejects  any  word  whose 
features  are  too  remote  from  those  stored  in  its  memory. 

Speech  recognition  can  be  divided  into  two  types.  The 
first  is  called  'isolated  speech"  or  "discrete"  recognition. 
Each  utterance  is  followed  by  a  0.1-0. 2  inter-word  pause. 
In  a  discrete  recognition  system,  the  "utterance"  is  not 
limited  to  Just  a  single  word,  for  example,  the  Threshold 
T-600  will  allow  utterances  up  to  two  seconds  in  length. 
The  second  type  is  called  "connected  speech"  recognition, 
simlliar  to  normal  human  speech  with  no  pauses  between 
words.  Utterances  In  connected  speech  units  can  thus  be 
made  up  of  many  individual  words  greeter  than  two  seconds  in 
duration. 

fchen  one  goes  from  computer  recognition  of  individual 
words  (isolated  speech),  to  full  sentences  as  in  natural 
speech  (connected  speech;,  tne  proclem  is  compounded  even 
mere.  In  isolated  speech  systems,  the  computer  can  clearly 
distinguish  between  consecutive  words.  In  contrast,  natural 
speech  wave  fcims  do  not  shew  clearly  where  one  werd  ends 


ar.a  the  next  begins,  and  the  time  alignment  problem  is  far 
more  difficult  with  connected  speech  than  with  isolated 
words.  Net  only  must  the  computer  time-align  individual 
words,  it  must  also  repeat  this  for  groups  of  words  or  whole 
sentences.  Cnee  an  input  word  is  recognized  in  connected 
speech  systems,  the  computer  must  also  decide  which  word  is 
most  likely  to  follow  the  input  word. 

2.  Value  of  Speech  Recognition  Systems 

Cochran  and  Riley  [Ref.  ?]  list  the  following 
advantages  to  ccmpucer  voice  entry: 

1.  Fast  learning  time 

2.  Eual  modes  cf  sensory  feedback 
2.  Not  restricted  to  one  language 

4.  Accuracy 

5.  S imul tanecus :  Use  of  hands,  use  of  eyes,  data  entry 

6 .  Automatic  and  multiple  processing 

7.  Remote  entry 

6.  Simple  data  entry 

The  learning  time  on  a  voice  input  device  is  shorter 
than  on  a  keyboard  or  typewriter,  and  accuracy  is  greater 
during  the  learning  period.  Because  spoken  language  is  a 
natural  part  of  life,  data  entry  using  voice  is  easy  once 
tne  individual  and  machine  are  trained.  The  auditory  and 
visual  feedback  immediately  reinforce  correct  input  and 
alert  the  operator  to  an  error  [Ref.  7:  p.  W0]  . 
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Because  the  machine  is  trained  tc  each  operator  s 
voice,  language  ana  pronunciation  are  not  critical  as  long 
as  the  operator  is  consistent.  This  means  that  regional 
accents  and  even  foreign  languages  are  acceptable  for  use 
with  the  voice  input  device  [Ref.  7:  p.  191]  . 

One  of  the  primary  advantages  of  the  voice  input  device 
is  that  it  frees  the  hands  and  eyes  for  other  tasks  while 
inputing  data. 

Voice  entry  can  be  useful  when  many  different  or  complex 
characters  need  to  be  entered  in  succession.  The  Threshold 
T-6ee  can  input  up  to  sixteen  characters  at  once  with  Just 
or.e  utterance. 

3 .  Limitations  of  Computer  Speech  Recognition 

Although  the  advantages  are  numerous  and  appealing, 
disadvantages  and  limitation  also  exist.  These  include: 

1.  Cost  -  Voice  data  entry  devices  vary  in  cost  from  $£00.00 

to  over  $60,000.00.  As  advances  are  made  in  high¬ 
speed  integrated  circuits,  costs  will  continue  tc 
lecrea  se . 

2.  Voice  input  can  be  overheard,  making  security  a  problem 

[Ref.  6]  . 

3.  Ambient  noise  interference. 

4.  Training  and  frustration. 

5.  Limited  vocabulary  -  The  user  is  limited  to  only  the 

vocabulary  which  has  been  loaded  into  computer  memory. 

c.  Changes  In  user  voice  characteristics  can 
retraining  [Ref  .  9]  . 
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As  in  aii  technology,  advantages  and  disadvantages 
exist.  The  final  test  is  always  the  marketplace.  In  both 
civilian  and  military  applications,  voice  units  are 
replacing  more  and  more  conventional  entry  devices. 
Cartography,  air  traffic  control  training,  aircraft  avionic 
systems,  and  military  command  and  control  centers  are  but  a 
few  of  the  present  users.  Industrial  users  are  improving 
production  line  and  shipping  productivity  with  these  devices 
by  as  much  as  40%  [Hef.  10].  The  Navy  is  utilizing  a  voice 
unit  at  their  Fleet  Ocean  Surveillance  Information  Center, 
CINCPAC  Fleet,  Honolulu,  Hawaii,  to  query  their  information 
lata  base.  General  Dynamics  recently  outfitted  an  F-16 
fighter  aircraft  with  a  piio t-operatea  voice  unit  for 
feasibility  tests. 

4.  CRT  lisplay  Techniques 

Two  major  classes  of  information  display 
technologies  exist  today:  the  cathode-ray  tube  (CRT)  and  the 
plasma  panel.  The  CRT  has  existed  since  the  1930's  while 
plasma  desplays  were  developed  in  the  late  1960's.  Both  use 
very  different  techniques  to  display  information. 

The  CRT  utilizes  an  electron  beam  to  excite  a 
phosphorous  coating  which  in  turn  emits  photons.  Images  can 
then  be  created  by  directing  the  electron  beam  over  the 
phospborous-coated  glass. 
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A  plasma  display  is  simiiiar  in  principle  to  a  neon 
tune.  It  consists  of  two  parallel  glass  plates  each  with 
many  parallel  conductors.  The  plates  are  separated  by  a  0.1 
millimeter  gap  filled  with  neon  gas.  A  voltage  between  the 
intersection  of  two  conductors  will  emit  a  glow.  Many  of 
these  points  of  light  can  he  made  to  form  images. 

The  B-520  displays  were  required  to  display  alphanumeric 
information  as  well  as  the  radar  presentation.  This  meant  a 
display  technique  capable  of  fast  updating,  which  the  plasma 
display  is  unable  to  do  [Ref.  11].  Thus,  the  type  of 
display  selected  for  use  on  the  E-52G  was  the  monochromatic 
CRT.  Methods  of  highlighting  information  on  a  monochromatic 
CRT  include: 

Hashing  -  Turning  displayed  Information  on  and  off  at 
approximately  3-5  Hz. 

Inverse  Video  -  A1 phanumerics  displayed  as  darn  characters 
over  a  white  background. 

location  -  Position  information  as  in  double-spacing. 
Highlighting  -  Achieved  by  overwriting  with  the  electron  gun 
thus  causing  the  image  to  stand  out. 

I.  STATEMENT  Of  THE  PROBLEM 

Part  I  of  this  thesis  evaluates  operator  performance  of 
the  flex  targeting  procedure  using  the  present  integrated 
keyboard  (IKE)  input  device  end  a  voice  input  device. 
Operator  speed  and  accuracy  were  used  to  evaluate  each 
system.  Would  voice  entry  improve  the  operator's 
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performance  as  defined  by  these  parameters?  Each  subject 
used  both  input  devices  ir.  leading  twenty  ALCM  target  sets 
per  device.  A  "target  set"  is  defined  as  the  data  comprised 
of  45  characters  the  ALCM  needs  leaded  in  its  memory  tc  fly 
to  and  locate  the  target  (Figure  11). 

Part  II  compares  the  way  in  which  this  inf ermat ion  is 
presently  displayed  (single-spaced)  on  the  navigator's  CRT 
with  twe  alternative  methods  of  presentation.  These  two 
alternatives  are  double-spacing  and  inverse-video. 
Characters  displayed  in  inverse-video  are  shown  as  dark 
figures  against  a  white  bacKground.  Normally,  characters 
are  displayed  as  white  characters  ever  a  black  background. 
Curing  the  inverse-video  portion  of  Part  II,  as  a  line  is 
updated,  that  line  is  "inverse-videoed"  when  reinserted  into 
the  target  set.  It  was  hoped  that  this  would  help  the 
operator  in  scanning  the  target  set  in  search  for  values  to 
oe  changed  out. 

A  capability  of  exercising  the  flex  targeting  option 
using  voice  input  might  s ignif lean tiy  reduce  bomber  exposure 
in  enemy  airspace,  while  giving  military  and  civilian 
leaders  more  time  for  retargeting  decisions  based  on 
reconnaissance,  changing  military  and  political  factors, 
etc.  The  method  of  displaying  the  flex  targeting  data  on 
the  CRT  could  also  have  a  similar  effect. 


HYPOTHESIS 

The  following  hypothesis  were  tested: 

1.  Hypothesis  Regarding  TIME 

H  :  There  is  no  difference  in  Time  to 

o 

enter  complete  target  sets  among 
the  two  entry  modes. 

Ej  :  Complete  target  sets  will  be  entered 
faster  using  the  voice  entry  mode. 

H  :  There  is  no  difference  in  Time  to 

o 

verify  and  correct  target  sets  among 
the  three  types  of  display  formats. 

Hj  :  Target  sets  will  be  verified  and  corrected 
fastest  using  the  inverse-video  format. 

2.  Hypothesis  Regarding  ACCURACY 

H  :  There  is  no  difference  in  the  Accuracy 

O 

of  target  sets  among  the  two  entry 
modes . 

:  Target  sets  will  be  entered  more 

accurately  using  the  voice  entry  mode. 

:  There  is  no  difference  in  Accuracy  in 
verifying  and  correcting  target  sets 
among  the  three  types  of  display  formats. 
Hj  :  Target  sets  will  be  most  accurately 

verified  and  corrected  using  the  inverse- 


II .  DESCRIPTION  OF  THE  EXPERIMENT 

A.  OBJECTIVES  AND  CONSTRAINTS 

The  purpose  of  this  experiment  was  twofold.  First,  to 
determine  whether  computer  voice  technology  was  better  than 
keyboard  entry  for  loading  tne  ALCM  flex  targeting  data. 
The  performance  of  ooth  entry  methods,  computer  voice  and 
the  integrated  keyboard  (IKE),  would  be  measured  by  subject 
input  speed,  accuracy,  and  time  versus  accuracy.  Second,  to 
determine  which  of  three  methods  of  data  presentation  on  the 
CRT  would  ce  test  suited  for  the  flex  targeting  procedure. 
The  performance  of  each  display  presentation  would  be 
determined  by  subject  input  speed,  accuracy,  and  time  versus 
accuracy  in  chan6ing  displayed  flex  targeting  data. 

An  effort  was  made  to  reconstruct  the  B-tcG  navigator's 
workstation  in  carrying  out  these  objectives.  Cnly  ALCI* 
related  controls  and  displays  were  incorporated  into  the 
mocgup  (see  Figure  2). 

B.  EQUIPMENT 

1 .  Navigator  Position 

For  the  purpose  of  this  thesis  we  will  assume 
through  some  ccmmunicat  lens  means  the  3-5EG  crew  can 
successfully  receive  new  ALCP  targeting  information  from 
higher  authorities.  Also  assumed  is  that  the  information 
can  be  readily  decoded  and  handed  to  the  navigator  in  paper 


E? 


form  which  will  then  oe  manually  entered  into  the 
appropriate  ALCF .  The  Keyboard  the  navigator  would  use  is 
unique  to  the  B-52G.  It  interfaces  with  the  Offensive 
Avionics  System  (OAS)  which  controls  both  navigation  ana 
weapons.  Both  navigator  and  radar  navigator  have  a  keytoard 
as  well  as  two  cathode-ray  tube  (CRT)  displays  for  each  of 
their  posticns  (see  Figures  3,4).  In  actuality,  either 
operator  can  change  ALCN  target  information.  But  in  most 
cases,  and  for  the  purpose  of  this  thesis,  the  navigator 
will  carry  out  the  procedure  (see  Table  I). 

In  simulating  the  operation  of  this  crew  position  at  the 
Human  Factors  Laboratory,  an  APPLE  II  "plus"  64K 
minicomputer  and  a  CONRAC  7x5  inch  CRT  display  were 
positioned  in  front  of  and  above  the  subject's  seat 
position.  A  keyboard  was  built  to  look  and  operate  like  the 
integrated  Keyboard  (IKB)  found  currently  in  the  B-52G. 

The  navigator's  position  dimensions  and  equipment 
location  for  this  experiment  were  obtained  from  the  OAS- 
ccnfigur^d  T-l£  navigation  simulator  located  at  Wright- 
Patterson  Air  Force  Base. 

2 .  Integrated  Keyboard  (IKB) 

In  order  to  measure  target  set  entry  times  by 
keyboard,  a  working  modei  of  the  IKB  used  in  the  B-52C-  was 
built.  Its  dimensions  ana  pertinent  keys  were  duplicated 
exactly  (see  i’lgure  5,6),  The  IKB  was  connected  via  a  Lear 
Siegler  Inc.  Arf"-3A  terminal  to  an  APPLE  II  minicomputer. 
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Equipment  Setup 


Figure  3.  Radar  Navigator/Navigator  Positions,  B-52G 


Radar  Navigator/Navigato 


The  A2M-3A  was  needed  oecause  the  APPLE  II  design  did  not 
allow  direct  connection  of  the  IO  to  the  APPLE  II 's 
Keyboard  plug  directly. 

3 .  Apple  II  Computer 

An  AFPLE  II  "plus"  64K  RAM  (Fandom  Access  Memory) 
rr inicomputer  was  used  to  control  all  video  displays  on  the 
CONRAC  CRT  (see  figure  ?).  As  each  subject  entered  target 
set  data,  the  information  was  stored  in  rain  memory  to  be 
printed  cut  after  the  session  using  a  MICRCLINE  Micron  80 
printer.  These  printouts  were  later  used  to  check  for  input 
errors  . 

4.  Voice  Recognition  System:  T60Z 

A  Threshold  Technology  Inc.  T-6ee  (hereafter 
referred  to  as  the  T600)  was  used  as  the  voice  recognition 
input  device  (see  Figure  8).  It  is  a  commercially  available 
device  capable  of  recognizing  up  to  £56  spoken  words  or 
phrases.  These  phrases  or  utterances  as  they  are  called 
must  be  limited  to  0. 1-2.0  seconds  in  length  [Ref.  12]. 

A  short  pause  of  at  ieast  0.1  second  is  required  between 
each  utterance.  Thus  the  T600  is  referred  to  as  an  isolated 
or  discrete  speech  voice  recognition  unit.  The  T620  voice 
recognition  system  consists  of  a  main  terminal  processor 
unit  containing  a  speech  preprocessor  and  microcomputer. 
The  remaining  components,  the  tape  cartridge  unit,  the 
microphone  preamplifier  and  display  (present  but  not  used 
during  experiment)  are  connected  separately. 
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gure  6.  Working  Model  of  Integrated  Keyboard  ( I  KB ) 


Figure  7.  APPLE  II  Computer  and  CONRAC  CRT 


The  microphone  used  was  a  noise  cancelling  SHURE  SM-10.  The 
T600  was  connected  to  the  APPLE  II  via  an  APPLE  serial 
interface  with  RS-232  standard  input/output . 

When  the  T600  receives  an  utterance  spoken  by  a  subject, 
the  voice  unit  will  react  in  one  of  two  ways.  The  utterance 
will  oe  compared  with  those  patterns  in  memory,  and,  if  a 
match  is  found,  the  corresponding  output  string  will  be 
transmitted.  The  alternative  is  that  no  match  will  be  found 
between  the  utterance  received  by  the  T600  and  that 
programmed  in  its  memory.  If  the  latter  occurs,  an  audible 
’beep"  is  sounaea.  with  no  output  string  transmitted.  This 
is  referred  tc  as  a  ccnreccehiticn.  It  can  result  from 
either  mispronunciation  of  the  utterance,  poor  initial 
training  of  the  word,  oackgrcund  noise  being  interpreted  by 
the  T6ee  as  an  utterance,  or  lack  of  T600  reliability. 

C.  SUBJECT  SELECTION  ANL  TRAINING 

1 .  Subject  Selection 

Subjects  used  for  this  experiment  were  students  and 
faculty  of  the  Naval  Postgraduate  School.  Four  were  female 
ar.d  sixteen  maie.  Ail  except  five  had  nc  previous 
experience  with  voice  entry  and  none  had  used  a  E-62G  IKE 
prior  to  this  experiment.  Seventeen  were  active  duty  U.S. 
military  officers  and  three  were  civilians. 
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2 .  T600  Vocabulary  Training 

Eefore  the  T 60£  could  be  used  as  an  input  device  in 
place  of  the  IKE,  voice  patterns  for  aii  input  commands  had 
tc  be  recorded  into  the  T600's  memory.  This  is  referred  to 
as  "training”.  Subjects  would  repeat  each  of  the  twenty 
vocaouiary  words  (see  Appendix  A)  ten  tires  into  the 
•ricrophcr.e.  The  microcomputer  within  the  T600  averages  the 
ten  utterances  into  one  digital  pattern  of  that  word.  This 
training  was  done  for  each  of  the  twenty  words.  Each 
vocabulary  word  had  a  corresponding  output  string.  Thus,  if 
the  T500  recognizes  an  utterance  as  "zero",  it  would  output 
a  "0",  and  so  on.  The  output  string  in  the  T60Z  can  be  up 
to  sixteen  characters  in  length. 

After  tne  twenty  words  were  trained,  each  was  repeated 
three  times  to  confirm  the  T 600  was  able  to  identify  them. 
If  a  mi srecogni ti on  or  a  nonrecognition  ("beep”  sound) 
occurred  2  out  of  3  tires  or  greater  for  a  given  word,  it 
was  retrained. 

3.  Integrated  Keyboard  Training 

Ail  subjects  had  never  seen  an  I  KB  be f ere 
participating  in  this  experiment.  As  stated  in  the  "Script 
And  Instructions  Tc  Subjects"  vsee  Appendix  2),  the 
operation  of  the  IK2  was  shown  to  each  subject  and  each  was 
allowed  unlimited  practice  usinw  a  sample  computer  program 
until  they  felt  corfortaole  with  the  IKE . 


D.  DEPENDENT  VARIABLES 

The  following  variables  were  calculated  for  each  group 
of  20  target  sets: 


%  Input  Accuracy  Typing  (IAT) 


*  Output  Accuracy  Typing  (CAT) 


%  Input  Accuracy  Voice  (IAV) 


%  Cutput  Accuracy  Voice  (CAV) 


NCE 

NCE 

+  NICE 

•  NME 

NCC 

NCC 

+  NIC 

+  NhC 

NCE 

NCE 

+  NICE 

+  NTE 

NCC 

NCC 

•*  NIC 

+  NMC 

100 


100 


100 


100 


Legend : 


Character 


NCC  : 

Number 

of 

Correc  t 

Characters  =  4E/target  set 

NmC  : 

Mum  ter 

cf 

hissing 

Characters  (san-e  as  KM) 

NIC  : 

hum  be  r 

of 

Incorrect  Characters 

NCE  : 

Number 

or 

Ccrrec  t 

Entries  =  68  max/terget  set 

Nr.E  : 

Number 

Cf 

missing 

Entries  (same  as  NmC ) 

NICE  : 

Number 

of 

Irco  rr e 

'  t  Entries 

-  Displayed  entries  which  are  ultimately  loaded 
into  the  ALCm. 


Entry  -  An  input  which  may  or  may  not  be  ultimately  loaded 
Into  the  ALCN. 
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1 1  I.  I  III  IHCIMR 


NCI 

%  Correct  Entries  =  - x  1 'l?. 

Nf-’E  +  TNE 

Legend:  NCI  :  Number  or  Correct  Entries  =  58  max/target  set 

NME  :  Number  cf  hissing  Entries  (same  as  NMC) 

TNI  :  Total  Number  of  Entries  = 

NCE  +  MICE  +  NEUTCE 

Character  -  Displayed  entries  which  are  ultimately  loaded 
into  the  ALCN. 

Entry  -  An  input  which  may  cr  may  net  be  ultimately  leaded 
into  the  ALCM. 

E.  EXPERIMENTAL  DESIGN  AND  PROCEDURE 

Each  subject  first  performed  Pert  1,  the  "Target  Loading 
Taslr",  in  which  he  first  entered  2?  ALCN  target  sets 
beginning  with  either  the  Integrated  Keyboard  ( I  KB )  or  the 
reeif  voice  input  device  (see  Appendix  C  for  subject  order). 
Fart  1  of  this  thesis  was  a  two-way  design  as  illustrated  in 
Figure  9.  Subject  order  with  the  two  conditions  was 
determined  using  ar.  ABBA  ordering  method.  This  was  repeated 
using  the  second  entry  method  with  the  same  target  sets. 
Performance  cf  each  subject  was  recorded  using  data  sheets 
shown  in  Appendix  P.  The  twenty  target  sets  used  in  Part  T 
are  contained  in  Appendix  I . 

Part  II  was  a  two-way  design  as  illustrated  in  Figure 
1C.  This  part  of  the  thesis  was  called  the  "Target 
Information  Verification  TaSK".  Here  the  subject  ves 
required  tc  examine  each  cf  eighteen  target  sets,  ordered  in 
three  groups  of  six  ,  which  had  ceen  preloaaed  by  the  author 
(see  Appendix  E )  . 
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VOICE 


IKB 


1 

-z 

S  4 

5 

U  6 

7 

B  6 

y 

J  10 

11 

E  12 

13 

C  14 

It 

t  ie 

17 

S  18 

19 

20 


TARGET  LOADING 
TASK 

CONCEPTUAL  DESIGN  '  PAET  I) 


Figure  y 


He  then  compared  each  ;c  a  list  in  which  various  characters 
haa  beer,  changed.  Changes  varied  from  0  to  9  characters  per 
set.  Seme  target  sets  required  nc  ccrrecticES,  while  others 
required  many  (see  Appendix  G).  The  subject  had  to  locate 
each  value  in  the  target  set  tc  ce  changed ,  and  maxe  the 
required  changes  (see  Figure  11).  Only  the  Integrated 
Keyboard  (IKB)  was  used  as  the  entry  device  for  Fart  II. 
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SINGLE-SPACE  DOUBLE-SPACE  INVZRSE-V IDEC 


2 

3 

S  4 

c 

U  6 

7 

e  a 

9 

J  10 

11 

E  12 

13 

C  14 

15 

T  16 

17 

S  18 

iy 
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TARGET  INiORMAT ION  VEPIFICATICN  TASK 
CONCEPTUAL  DESIGN  (PART  II) 


iigure  10 


Each  of  the  three  groups  of  six  target  sets  required  a  tctai 
of  Z6  randomly  distributed  changes  per  group,  group.  The 
first  set  of  six  target  sets  was  displayed  in  a  single- 
spaced  format  as  is  presently  acne  in  the  B-52G  <£ig.  11). 
The  second  set  of  six  was  displayed  in  aoutle-sraced  forr. 
The  third  set  cf  six  was  displayed  single-spaced,  but  as  a 
line  was  modified,  that  line  then  became  displayed  in 
inverse-7idec  form.  Characters  displayed  in  inverse-video 
are  shown  as  dartc  figures  against  a  white  hack-ground.  For 
details  of  the  experimental  procedure,  see  Appendix  B. 
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'  FLEX  TARGETING 


i  i  i  i  i  i  .  i  i  i  '  i  i  i  i  i  i  i  i  i  i  i  i  i  i  i  i 
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1  N00000200  < - target  latitude 

F000000000  < - target  longitude 

0000000000  < - \ 

00002 000000  < - \ 

00000  < - \fligdtpath  i ni’crmat ior 

required  cy  ALCM 


f-CDIFY 


N000000000 

- cursor 


< - line  #1  ready 

tc  be  trcdlfied 


I 

I 

t  I  I  1  I  I  I  I  I  M  I  1  I  I  I  I  I  I  I  I  I  I  I  I  M  II  (  I  I  1  1  »  I  I  I  1  I  M  I  f  l  i  I  {  l  I  \  I 

'  !  »  I  I  ■  1  I  I  '  I  \  I  I  I  t  1  ‘  i  I  I  I  I  I  !  i  I  I  »  I  !  •  I  »  ■  \  t  \  S  ;  ,  »  |  I  I  |  I  !  I  I  l  ! 


TARGET  SET  FCRl^AT  CURRENTLY  USED  EOR  FLEX  TARGETING 
(Adapted  Ytct  Reference  6) 

FIGURE  11 
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TAhLS  I 


RETARGET ING  PROCEDURE 


ACTION  RESPONSE 


Desired  ALCM  is  selected  1.  CRT  in  front  of  nav- 

oy  navigator.  igatcr  displays  target 

inf o rrra ti on  as  shown 
in  Figure  11. 

Line  #1  is  selected  to  be  2.  Line  #1  is  repeated  at 
changed.  bottom  of  CRT  ready  to 

be  changed  as  shown  in 
Figure  11. 

New  data  is  entered  using  3.  New  line  *1  appears  at 
Keyboard  for  line  »1 .  bottom  of  CRT  as  it  is 

typed,  overwriting  the 
original  line  #1. 

New  line  #1  is  completed;  4.  Old  line  £1  is  replaced 
"ENTER"  button  is  pressed.  by  new  line  #1. 

Line  42  is  seiected  tc  be  t.  Line  42  is  repeated  at 
changed .  bottorr  of  CP.T  ready  to 

be  changed. 


This  procedure  Is  continued  until  all  five  lines  have  beer, 
changed  out.  After  the  last  line  (  #5  )  has  been  entered, 
the  ALChi  is  ready  for  launch  against  its  new  target. 


In  Parts  I  and  II,  subjects  called  up  each  target  set  as 
is  done  in  the  E-52G  using  a  letter-number  comb inat  ion .  The 
six  ALC^s  stored  under  the  left  aircraft  pylon  are  referred 
tc  as  A1  through  A6 .  The  eight  stored  in  the  bent  bay  are 
referred  to  as  El  through  Be.  The  six  ALCMs  stored  under  the 
right  aircraft  pylon  are  referred  tc  as  Cl  through  C6. 
Figure  12  shows  pictoraily  the  identification  and  position 
of  each  ALCM . 


time*# 


III  .  RESULTS 

A.  INTRODUCTION 

Results  fer  Fart  I  (Target  Leading  Task)  and  Part  II 
(Target  Information  Verification  Task)  will  be  discussed 
separateiy.  For  each,  the  results  are  subdivided  into  Time, 
Input  and  Output  Accuracies,  and  Tirre  versus  Accuracy.  Paw 
data  analyzed  in  this  section  is  contained  in  Appendixes  H 
and  I . 

5.  RESULTS  FOP  PART  I 
1 .  Tirre 

Figure  13  is  a  plot  showing  the  tirre  required  for 
each  subject  to  load  ali  twenty  target  sets.  Keyboard  entry 
was  faster  than  the  voice  entry  technique  for  fifteen  of  the 
nineteen  subjects.  Nc  difference  was  r.cted  in  the  ti^e  tc 
complete  using  either  technique  for  one  subject,  while  voice 
entry  was  faster  for  turee  subjects.  The  five  subjects  at 
the  far  right  of  the  graph  had  previous  experience  with 
vc ice  entry.  Analysis  cf  Figure  13  indicates  no  marked 
difference  among  those  with  prior  voice  entry  experience. 
Mean  and  standard  deviation  fer  the  voice  data  are  27.56  and 
2.42  respectively.  Mean  ana  standard  deviation  for  the  IK? 
data  are  24.64  and  5.1b  respect i veiy . 
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TABLE  II 


ANALYSIS  CF  VARIANCE:  SUEJECT  BY  TIME  -  PART  I 


SOURCE 

SS 

df 

MS 

f 

P 

Between  Subjects 

412.43 

19 

21.71 

1.32 

ns 

Between  Methods 

£5  .24 

1 

65.24 

5.16 

<.05 

Error 

312. £4 

19 

16  .47 

Tcta  i 

£10.52 

39 

The  Integrated 

Keyboard 

input 

method 

proved 

to  be 

$ ignificantiy  quicker  than  the  voice  method  as  shewn  in 
Table  II  (P  <.Z5)  . 

2 .  Input  Accuracy 

Eigure  14  shews  Subject  by  Percent  Input  Accuracy. 
The  IK3  percent  input  accuracy  was  higher  for  74  percent  of 
the  subjects.  Vcice  was  tetter  for  26  percent.  This  was 
confirmed,  using  an  Analysis  of  Variance  shewn  in  Table  III. 
I  KB  input  accuracy  was  significantly  better  '.?  <  .01)  than 
vcice  entry.  (In  oraer  to  reet  requirements  regarding  the 
distributions  cf  the  lata,  an  arc-sir.e  transformation  was 
applied  to  the  percent  input  accuracies  prior  to  performing 
the  Analysis  cf  Variance.) 


TABLE  III 


ANALYSIS  OF  VARIANCE :  SUBJECT  BY  INPUT  ACCURACY  -  PART  I 


SOURCE 

SS 

df 

MS 

V 

p 

Between  Subjects 

0.5759 

19 

0.2303 

1  .31 

Between  Methods 

0  .2528 

1 

0.2538 

10.99 

<.01 

Error 

0.4383 

19 

0.0231 

Total 

1 .  £680 

39 

3.  Output  Accuracy 

Figure  lb 

shows  Sub 

ject 

versus 

Percent 

Output 

Accuracy.  Analysis 

or  Variance 

in  Table 

IV  confirmed  no 

significant  difference  tetween  voice  and  IKB  percent  output 
accuracies  existed.  (In  order  to  meet  requirements 
regarding  the  distributions  of  the  data,  an  arc-sin® 
transformation  was  applied  to  the  percent  output  accuracies 
pricr  to  performing  the  Aral/sis  of  Variance.) 


Percent 

Input 

Accuracy 


V 

K 

E 

* 


Input  Accuracy  Vcice 
Input  Accuracy  Keyboard 
Voice  In.Acc.  =  Keyboard  In.Acc 
Previous  Experience  Vith  Vcice 


SUBJECTS 


iigure  14.  Input  Accuracy  by  Subject 


SOURCE 

SS 

df 

MS 

F 

P 

Between  Subjects 

0 .7946 

19 

0.0416 

<1 

n  s 

Between  Methods 

0.0214 

1 

0.0214 

<1 

ns 

Error 

1 .0596 

19 

0.0556 

Total 

1 .6755 

29 

4.  Time  Versus 

Accuracy 

Figures  16 

through 

19  S 

how  Input 

and 

Output 

Accuracies  plotted 

against  Time  for 

both  voice 

and 

IO  entry 

methods.  Figure  16  shows  no  apparent  tradeoff  between 
Fercent  Input  Accuracy  and  Time  for  voice  entry.  Hcwpver, 
Figure  1?  througn  iy  snow  pronounced  increases  in  accuracies 
as  time  to  complete  the  tasi  Increased. 


\ 

\ 


w 


-  V  =  Output  Accuracy  Vcice 

Percent  -  K  =  Output  Accuracy  Keyboard 

Output  B  =  Voice  Out.Acc.  *  Keyboard  Cut.Acc. 

Accuracy  *  =  Previous  Experience  With  Vcice 
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SUBJECTS 


figure  15.  Output  Accuracy  by  Subject 
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C.  RESULTS  ECR  PART  II 
1.  Time 

Figure  20  shows  average  times  for  each  of  the  three 
display  formats  tested.  Analysis  of  Variance  between  the 
three  lisplay  formats  shewn  in  Table  V  Indicated  no 
significant  differences.  However,  a  significant  difference 
between  subjects  (P  <'.2l)  was  detected. 


TABLE  V 

ANALYSIS  OF  VARIANCE:  TirE 
BY  DISPLAY  IOBMATS  -  PART  II 


SOURCE  SS 

Between  Subjects  37.52 

Between  Methods  1.15 

Error  16.  76 

c.  c  ,  4 


If 

PS 

V 

p 

19 

1.97 

4.46 

<•01 

2 

0.57 

1 .30 

ns 

36 

0.44 

Total 


59 


Average 

Time 

(minutes ) 
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Display  Format 


Figure  22.  mean  Time  Ana  Standard  Deviation  tc 
Complete  Update  Using  Fach  Display  Format 


Figure  21  shows  Mean  Input  Accuracy  plotted 
against  each  Display  icrtnat.  No  significant  difference  was 
found  between  display  fortrats  as  shown  in  the  Analysis  of 
Variance  in  Table  VI.  (In  order  tc  rreet  requirements 
regarding  the  distributions  of  the  data,  an  arc-sine 
transformation  was  applied  tc  the  percent  input  accuracies 
prior  to  performing  the  Analysis  of  Variance.) 


TABLE  VI 

ANALYSIS  CF  VARIANCE:  SUBJECT  BY 
INPUT  ACCURACY  -  PART  II 


SOURCE 

SS 

df 

MS 

V 

P 

Between  Subjects 

0 .y40e 

IS 

0.0495 

1.10 

ns 

Eetween  Methods 

0  .0233 

2 

0.0116 

<1 

ns 

Error 

1 .7105 

3e 

0.0450 

FT 

I 


i 

I 
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Mean 
Inpu  t 
Accuracy 
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i  i 
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Figure  21.  Mean  Input  Accuracy  And  Standard  Deviation 
For  Each  Display  Format 
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3 .  Output  Accuracy 

Figure  ZZ  shews  Mean  Output  Accuracy  plotted 
against  each  Display  Format.  No  significant  differences 
were  computed  as  shewn  in  the  Analysis  of  Variance  in  Table 
VII.  (In  order  to  meet  requirements  regarding  the 
distributions  of  the  data,  an  arc-sine  transformation  was 
applied  to  the  percent  output  accuracies  prior  to  performing 
the  Analysis  of  Variance.) 


TABLE  VII 

ANALYSIS  CF  VARIANCE:  SUBJECT  BY 
OUTPUT  ACCURACY  -  PART  II 


SOURCE 

SS 

If 

MS 

F 

p 

Between  Subjects 

2  .610 

ly 

0.136 

1  .64 

n  s 

Between  Methods 

0.296 

2 

0.046 

<1 

ns 

Error 

3 .  i8y 

36 

0.064 

Total 

6  .696 

69 

rm 
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Figure  22.  Pean  Output  Accuracy  And  Standard  Deviation 
For  Sach  Display  Format 


4 .  Time  Versus  Accuracy 

Analysis  of  Variance  for  subject  tinges  and  input  and 
output  accuracies  shewed  nc  significant  differences. 
However,  plots  of  Percent  Input  Accuracy  versus  Tirre  for  the 
three  display  formats  indicate  the  dcu tie-space  format 
resulted  in  the  best  percent  input  accuracy  over  tirre. 
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Tire  (tr  in  .  ) 

Figure  23.  Percent  Input  Accuracy  by  Tin-e 
(S ingie-Spe  ce ) 


Figures  23  and  25  show  data  scattered  over  wider  ranges  as 
compared  with  Figure  24  wnich  shews  good  clustering  above 
the  95  percent  level. 
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ligure  i:4.  Percent  Input  Accuracy  by  Titre 
(Double-Space ) 
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Percent 

Input 

Accuracy 


T ime  (min  . ) 

Figure  25.  Percent  Input  Accuracy  by  Time 
( Inverse-V ideo ) 


An  examination  of  the  Percent  Output  Accuracy  by  Time 
plcts  shewed  no  significant  grouping  of  the  data  in  all 
three  display  formats. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  GENERAL 

The  purpose  of  this  thesis  has  been  twofold.  Part  I 
investigated  the  use  cf  a  discrete  voice  entry  system  in 
loading  Air  Launch  Cruise  Missile  (ALCM)  target  information. 
A  direct  comparison  was  made  between  voice  entry  and  the 
integrated  Keyboard  (IKE)  presently  operational.  Time  to 
load,  input  and  output  accuracies,  and  time  versus  accuracy 
measurements  were  recorded  for  each  of  the  twenty  subjects. 

Part  II  investigated  three  means  of  displaying  the  ALCM 
target  information  on  a  CRT.  Information  was  updated  using 
single-spaced ,  double-spaced,  and  inverse-video  formats. 
Only  the  IKE  was  used  in  Part  II.  Time  to  update,  input  an 
output  accuracies  and  time  versus  accuracy  measurements  were 
recorded  for  each  subject. 

B.  CONCI JSICNS 

1 .  Part  I 

It  had  been  hypothesized  that  target  sets  would  be 
entered  faster  using  the  voice  entry  method.  However,  the 
IKE  was  significantly  faster  (P  <  .05)  than  the  voice 
method.  It  was  suspected  that  this  may  have  teen  due  in 
part  to  first  time  use  of  the  voice  entry  device  by  fifteen 
out  cf  twenty  subjects.  Due  to  experimental  time 
constraints,  subjects  had  at  most  15-20  minutes  to  practice 
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with  voice  entry  prior  tc  running  the  experiment.  However, 
analysis  of  Figure  13  shows  that  only  two  out  of  the  five 
subjects  with  prior  voice  entry  experience  accomplished  data 
entry  quicker  using  voice.  Thus,  prior  exposure  to  voice 
did  not  improve  subject  entry  times.  It  should  he  noted 
that  despite  slower  input  speeds  using  voice  entry,  60 
percent  of  the  subjects  preferred  voice  entry  use  over  the 
I  KB . 

It  should  be  noted  that  a  discrete  voice  unit  was  used 
for  this  experiment.  However,  a  "connected  speech" 
recognition  unit  would  have  oeen  more  appropriate  for  this 
taslc,  and  probably  would  have  shewn  faster  input  times.  A 
connected  speech  system  is  able  to  receive  multiple  Inputs 
at  cnce,  ie.,  "1-4-3-2-5" ,  with  no  pause  between  utterances. 
This  more  natural  speech  would  be  more  comparable  with 
Keyboard  entry  than  the  discrete  voice  entry.  Unf ortunately 
such  a  system  was  not  available. 

It  was  hypothesized  that  target  sets  would  be  entered 
more  accurately  using  the  voice  entry  method.  The  IKB  was 
significantly  higher  in  input  accuracy  (P  <  .01)  than  voice 
entry.  Thus  fewer  corrections  were  made  to  data  on  the 
screen  before  committing  it  tc  the  weapon  using  the  IKE 
entry  technique  rather  then  voice  entry. 

Another  factor  which  effected  input  accuracy  was  the 
training  of  the  voice  unit  by  the  subject.  E»ch  utterance 
was  repeated  three  times  in  order  to  detect  poor  training. 
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If  the  voice  unit  had  problems  recognizing  an  utterance 
either  after  training  or  during  practice,  that  word  was 
retrained.  In  many  instances  it  was  not  noticed  until  well 
into  the  experiment  that  certain  words  had  either  teen 
pccrly  trained  or  were  being  pronounced  differently  by  the 
subject.  Changes  in  pronunciation  may  also  be  attributed  to 
the  stress  impcsea  by  the  experiment  which  was  not  present 
during  the  vocabulary  training  session.  Regardless  of  the 
explaination,  subjects  made  less  errors  using  the  IKE  method 
of  entry. 

No  significant  difference  in  the  output  accuracy  existed 
between  input  devices.  This  indicates  the  data  was  checked 
equally  well  by  each  subject  prior  to  entering  the  data  and 
moving  on  tc  the  next  weapon,  irrespective  of  entry 
technique . 

No  apparent  tradeoff  was  uoted  between  Percent  Input 
Accuracy  and  Time  for  voice  entry  isee  figure  15).  However, 
pronounced  increases  in  accuracy  were  noted  for  Percent 
Input  Accuracy  by  Time  using  the  IKB  (see  figures  17). 

Possible  fatigue  was  suspected  during  the  voice  entry 
portion  in  Part  I.  Subsequent  analysis  of  all 
misrecognitions  and  nonrecognitions  over  the  twenty  target 


sets  shewed 

an  increase 

in 

average  errors 

as  the  the 

experiment 

progressed , 

but 

this  increase 

was  not 

statistically 

significant 

(see 

Table  XIII) . 

6y 

% 


l 


2.  Part  II 


It  was  hypothesized  that  target  sets  would  be 
verified  and  corrected  quickest  and  most  accurately  using 
the  inverse-video  format.  Analysis  showed  no  significant 
differences  in  time,  input  or  output  accuracies,  and  time 
versus  accuracy  between  the  three  display  formats  tested. 
This  was  rather  suprising.  It  was  originally  telieved  the 
single-space  format  would  require  much  mere  time  and  induce 
more  errors  than  tne  other  two  methods.  There  were 
differences  in  subject  preference  among  the  three  display 
formats.  The  double-space  format  was  preferred  by  65 
percent,  as  compared  with  20  percent  for  the  inverse-video 
and  15  percent  for  the  single-space. 

C.  RECOMMEN IATIONS 

Based  on  the  data  in  Part  I  it  can  be  concluded  that 
while  voice  entry  is  definitely  preferred,  the  IKE  is 
superior  in  entry  speed  and  input  accuracy.  Due  to  limited 
resources,  this  thesis  had  only  a  discrete  voice  entry 
device  to  measure  against  an  IKB.  It  is  the  author's 
opinion  that  a  ccnected  voice  unit  would  have  shown  much 
greater  input  speeds  than  the  discrete  unit  used  here. 
Thus,  voice  applications  in  areas  requiring  quick  leading  of 
single  characters  should  Iook  toward  the  continuous  voice 
systems  for  improvement  over  conventional  keyboard  entry 
devices . 
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While  there  was  no  significant  difference  among  the 
three  display  formats,  it  is  suggested  that  the  aoubie-space 
display  format  be  used.  Use  of  this  format  would  incur  no 
additional  cost  and  would  satisfy  user  preference. 

It  might  be  desirable  tc  compare  the  effectiveness  c? 
the  three  display  formats  under  subject  posed  and 
experimenter  posed  conditiors.  Thus,  the  effect  of  stress 
could  be  simulated. 

A  brief  examination  of  nonrecognitions  in  Table  XIV 
indicated  that  the  terms  0,  Z,  4,  end  5  were  more  often  not 
recognized  than  the  other  numerical  terms.  A  x1  »« 
performed  on  the  frequencies  observed  for  the  0  through  y 
entries  and  a  %  value  of  137.53  was  obtained  (10  df,  P  < 
.001).  The  0  through  9  values  had  been  drawn  from  a  table 
of  random  numbers,  thus  each  had  an  equal  probability  of 
being  misreccgnized .  The  data  suggests  that  the  entries 
0,  2,  4,  and  5  are  over-represented  while  the  entries  9  and 
3  are  under-represented. 
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APPENDIX  A 


T600  VOCABULARY  LIST 


T600 

T600 

Location 

Register 

Output 

000 

zero 

001 

one 

002 

two 

003 

three 

004 

four 

005 

five 

006 

six 

007 

seven 

008 

eight 

009 

nine 

010 

control 

011 

irodify 

012 

enter 

013 

yes 

014 

no 

015 

al  fa 

016 

bravo 

017 

Charlie 

018 

left 

019 

right 
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APPENEIX  B 


SCRIPT  &  INSTRUCTIONS  TO  SUBJECTS 

1)  The  CRT  and  the  integrated  Keyboard  (IEB)  you  see 
here  are  slmiliar  tc  these  found  at  the  navigator's  position 
in  the  B-52G .  I  will  be  measuring  your  ability  to  load 
target  information  in  actual  ALCM  formats  using  keyboard  and 
voice  entry.  The  experiment  is  divided  into  two  parts. 
Part  I  will  require  you  to  load  20  ALCM  target  sets  using 
both  the  IKB  and  voice  entry.  Part  II  will  require  you  to 
call  up  each  of  six  ALCM  target  sets  I  have  preleaded,  and 
irate  corrections  to  them  if  necessary  with  new  values  you 
will  have  in  front  of  you.  This  will  be  done  three  times, 
each  with  a  different  CRT  display  format. 

2)  Show  photos  (Figures  3,4)  of  navigator's  position  in 
E-S2G.  Show  actual  IKE  (Figure  5;  and  the  working  model 
used  in  experiment  (Figure  6). 

3)  If  the  voice  unit  is  used  first,  train  the  20 
vocabulary  words  at  this  time.  Explain  the  theory  of  voice 
entry.  After  words  have  been  trained,  have  subject  repeat 
each  three  times,  confirming  the  voice  unit  can  understand 
at  least  two  out  of  the  three  utterances.  For  each  word 
falling  the  above  test,  have  subject  retrain  that  word. 

4)  Show  subject  typical  AICM  target  sets.  Run  practice 
computer  program.  Show  subject  how  to  call  up,  enter, 
modify,  etc., 


target  sets. 


You  have  as  iruch  titre  as  you  feel  is  necessary  to 
practice.  When  you  feel  corrfortabie  with  the  procedure,  we 
will  proceed  with  Part  I  cf  the  experiment. 

(Answer  any  questions  and  assist  if  necessary.) 

5)  You  will  be  given  20  ALCM  target  sets  printed  cn  3 
sheets  of  paper.  When  I  say,  "ready,  begin",  enter  the  sets 
as  fast  as  possible  while  maintaining  accurate  inputs.  If 
you  make  a  mistake,  be  sure  to  correct  it. 

Any  questions?  Reedy,  begin. 

6)  Repeat  above  paragraphs  (4)  and  (?)  using  other 
entry  method,  after  subject  completes  a  ten  minute  break. 

?)  We  will  now  begin  Part  II  of  the  experiment.  I  have 
preloaded  18  ALCM  target  sets  in  three  groups  of  six.  You 
will  be  given  each  set  of  these  six  target  sets  separately 
on  a  sheet  cf  paper  with  seme  cf  the  values  changed.  You 
will  be  required  to  display  each  target  set  on  the  CRT, 
compare  it  to  the  corresponding  target  set  cn  the  paper,  and 
make  the  necessary  corrections  to  the  target  set  on  the  CP.T. 
Some  target  sets  will  require  no  corrections,  while  others 
will  require  many.  Each  group  of  six  target  sets  will  be 
displayed  on  the  CRT  in  one  of  three  ways:  single-  spaced, 
dcuble-spaced,  or  inverse-video. 

Show  subject  example  of  ALCM  target  sets  with  changed 
values  on  a  sheet  of  paper.  Run  practice  computer  program. 
Allow  subject  to  practice  until  he  Is  comfortable  with  the 


procedure . 


You  have  as  rruch  tire  as  you  feel  is  necessary 


to 


practice.  When  you  feel  comfortable  with  the  procedure,  we 
will  proceed  with  Part  II  of  the  experiment • 

(Answer  any  questions  and  assist  if  necessary.) 

fa)  Repeat  item  #7  if  subject  fails  to  understand  Part 
II  of  the  experiment. 

If  you  make  a  mistake,  be  sure  to  correct  it. 

Any  questions?  Ready,  Degin. 

y)  Repeat  above  paragraphs  (7)  and  (£)  using  other 
entry  method. 


APPENDIX  C 


EXPERIMENTAL  ORDER  FOR  SUEJECTS 


SUBJECTS  1,3,6,7.9,11,13,15,17,19  performed  Pari  II  of  the 
experiment  in  the  following  order: 

Target  Leading  Task  -  Voice, hey ccard 

SUEJECTS  2 ,4 ,6 ,8 , 10 ,12 , 14 ,16 ,1c ,20  performed  Part  II  of  the 
experiment  in  the  following  order: 

Target  Loading  Task  -  Keyboard .Voice 

SUBJECTS  1, 2, 3, 6, 6, 7, y, 11, 13  performed  Part  II  of  the 
experiment  in  the  following  order: 

Target  Information  Verification  Task  - 
Single-Spaced,  Double-Spaced,  Inverse-Video 

SUBJECTS  15,17,19,4,6  performed  Part  II  of  the  experiment  in 

the  following  order: 


Target  Information  Verification  Task  - 
Double-Spaced,  Inverse-Video,  Single-Spaced 

SUBJECTS  10, 12, 14, 16,  IS, 20  performed  Part  II  of  the 

experiment  in  the  following  order: 


Target  Information  Verification  Task  - 
Inverse-Video,  Single-Spaced,  Double-Spaced 

Note:  All  twerty  subjects  performed  first 
Part  I,  "Target  Loading  Task".  This 
was  followed  by  Part  II,  "Target 
Information  Verification  Task*. 
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APPENDIX  D 

SUBJECT  DATA  SHEETS 
TARGET  LOADING  TASK  DATA  SHEET 

NA(*E :  _  RANK: _ 

DATE:  _  SERVICE: _ 

ACCURACY 

IAT _ 

OAT 

IAV _ 

OAV _ 

VOICE  or  KEYBOARD 

N^E  =  NMC 

NCE 

TNE 

%  Correct;  Entries: _ 


7? 


TARGET  LOADING  TASK  DATA  SHEET 


CIRCLE  ONE:  VOICE 


KEYBOARD 


NI*C  ]  TIPE  !  NICE 
NME  ]  min.  ! 


NEUTCE 


line# 


#entrles 


TARGET  INFORMAT  ION  VIRIFI  CATION  DATA  SHEET 


TARGET  INFORMATION  VERIFICATION  DATA  SHEET 


#  Incorrect  Char.!  NIC  ]  TIME  j  NICE  j  NEUTCE  | 
to  be  changed  out!  ]  min.  !  !  ! 


TARGET  INFORMATION  VERIFICATION  EATA  SHEET  (cont.) 


VEA  .# 


#  Incorrect  Char, 
to  be  changed  out 


NIC 


TIME 
min . 


NICE 


NEUTCE 


I 

I 

I 

I 
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APPENDII  E 


TARGET  LOADING  TASK  INPUT  DATA  LIST 


WEAPON  A1  :  1  N31864725 

2  E 467214085 

3  1965430873 

4  97031687439 

5  61498 


WEAPON  A2  :  1  N 19857435 

2  E973687525 

3  1210934753 

4  53272361071 

5  40912 


WEAPON  A3  :  1  N56672130 

2  E 120957264 

3  1295725482 

4  07362756364 

5  28158 


WEAPON  A4  :  1  N82659374 

2  E598326188 

3  3871946572 

4  28386610864 

5  16452 


WEAPON  A5  :  1  N 10725497 

2  E728552547 

3  1650724572 

4  75189473575 

5  26836 


WEAPON  A6  :  1  N 16500364 

2  E660375724 

3  0734712641 

4  16406437553 

5  65838 


83 


WEAPON  Bl 


WEAPON  B2  : 


WEAPON  B3  : 


WEAPON  B4  : 


WEAPON  B5  : 


WEAPON  £6  : 


WEAPON  B?  : 


1  M 17696354 

2  E194653853 

3  5481937906 

4  64868351041 

5  86357 


1  N 52849654 

2  E107452749 

3  1075382657 

4  63956272651 

5  20756 


1  N 13474638 

2  E 385638361 

3  6120346246 

4  01852954752 

5  65820 


1  N01472863 

2  E 184644278 

3  1038565275 

4  07452865375 

5  15832 


1  N86612086 

2  E68 1902642 

3  1943028474 

4  02857918491 

5  65821 


1  N 10238475 

2  E027361856 

3  7453298136 

4  08164829154 

5  68264 


1  N02746382 

2  Z57S103856 

3  5482107462 

4  10825483217 

5  62741 
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WEAPON  B8  : 


WEAPON  Cl  : 


WEAPON  C2  : 


WEAPON  C3  : 


WEAPON  C4  : 


WEAPON  C5  : 


WEAPON  C6  : 


1  N 58286108 

2  E 193656828 

3  1037561937 

4  16401746218 

5  51073 


1  N 10354826 

2  E 102462651 

3  6295016472 

4  39741845226 

5  10275 


1  N 17494625 

2  E629575107 

3  4610374628 

4  10617381561 

5  19565 


1  N  56194015 

2  E452808462 

3  9108265726 

4  51072658254 

5  75382 


1  N01735473 

2  E107562743 

3  1656264634 

4  51956271064 

5  61028 


1  N 10741970 

2  E963557254 

3  5629601652 

4  51036582514 

5  62814 


1  N86952753 

2  E1063S5672 

3  1653903647 

4  01758265781 

5  01536 
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APPENDIX  F 


TARGET  INFORMATION  VERIFICATION  TASK 
PRELOADED  TARGET  SETS 


Summary  of  Characters/ 1  ine  to  be  changed  out: 


Seven  hundred  seventy-four  total  characters  are  possible 
to  be  changed  out  in  all  18  target  sets.  A  value  of  10%  was 
chosen  to  be  changed  out  by  each  subject.  Ten  percent  of 
774  =  78,  the  number  chosen  to  be  changed  out. 

All  eighteen  target  sets  were  divided  into  three  groups 
of  sii  target  sets/display  method.  Each  of  the  three  groups 
had  26  characters  tc  be  changed  out.  Characters  to  be 
changed  out  were  selected  randomly. 


PCharacters/line  to 
be  changed  out: 


WEAPON  A1  :  1  N02746382  5 

2  E578103856  0 

3  £462107482  1 

4  10825483217  0 

5  62741  0 

WEAPON  A2  :  1  N 10354826  1 

2  E 102462651  1 

3  6295016472  0 

4  39741845226  1 

5  10275  e 


#Characters/line  to 
be  changed  out: 


WEAPON 

A3  : 

1 

N82659374 

0 

2 

E5983261S8 

5 

3671946572 

3 

4 

£8386610864 

0 

5 

18452 

0 

WEAPON 

A4  : 

1 

N 31864725 

0 

2 

E467214085 

0 

T 

V 

1965430673 

1 

4 

97031687439 

2 

5 

61498 

0 

WEAPON 

At  : 

1 

N 13474638 

1 

2 

E385638361 

2 

■» 

w 

6120346246 

1 

4 

01852954752 

1 

5 

65620 

1 

WEAPON 

A  6  : 

1 

N0147 2863 

0 

2 

E 164644278 

0 

■j 

1038565275 

0 

4 

07452865375 

0 

K 

w 

15832 

0 

WEAPON 

£1  : 

1 

N 17494625 

1 

m 

E629575107 

0 

w 

4C10374628 

2 

4 

10817381561 

0 

5 

19565 

0 

WEAPON 

£2  : 

1 

N56672130 

1 

2 

S120957264 

2 

3 

l£y5725482 

0 

4 

07362756364 

1 

5 

28158 

0 
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#Characters/line  to 
be  .changed  out: 


WEAPON  B3  :  1  N 10238475  2 

2  E027361656  7 

3  7453298136  0 

4  08164829154  0 

5  68264  0 

WEAPON  B4  :  1  N86952753  0 

2  E106385672  0 

3  1653903647  1 

4  01758265781  1 

5  01538  2 

WEAPON  B5  :  1  N 17494625  1 

2  E629575107  0 

3  4610374628  2 

4  10817381561  0 

5  19565  0 

WEAPON  E6  :  1  N 19857435  3 

2  E973687525  0 

3  1210934753  0 

4  53272361071  0 

5  40912  0 

WEAPON  B7  :  1  N 17896354  1 

2  E 194653853  6 

3  5481937906  2 

4  64868351041  0 

5  86357  0 

WEAPON  B8  :  1  N58286108  4 

2  E 193656828  1 

3  1037561937  0 

4  18401746218  0 

5  51073  0 
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#Characters/line 

be  changed  out 

WEAPON  Cl  :  1 

N01472863 

0 

2 

E1S464427B 

0 

T 

w 

1038565275 

e 

4 

07452665375 

0 

C 

15:832 

0 

'WEAPON  C2  :  1 

N 10725497 

2 

2 

E728552547 

1 

T 

C 

1650724572 

0 

4 

75189473575 

0 

5 

26836 

3 

WEAPON  C3  :  1 

N52849654 

0 

2 

E10745274y 

1 

3 

1075382657 

0 

4 

63956272651 

0 

5 

20756 

0 

WEAPON  C4  :  1 

M 56194015 

0 

2 

1452806462 

2 

3 

9108265726 

0 

4 

51072658254 

0 

e, 

75382 

3 

ey 


APPENDIX  G 


REVISED  1ARGET  SETS  TO  BE  UPDATED  IN  PART  II 
TARGET  INFORMATION  VERIFICATION  TASK 
SINGLE-SPACED  DISPLAY  FORMAT 


WEAPON  A1  : 


1  N 07342394 

2  E578103856 

3  5082107462 

4  10825483217 
£  62741 


WEAPON  A2 


WEAPON  A3 


WEAPON  A4  : 


1  N 10254826 

2  E102462671 

3  6295016472 

4  09741845226 

5  10276 


1  N826E9374 

2  E758826381 

3  3773946571 

4  28366610664 

5  18452 


1  N31864725 

2  E467214085 

3  1965430473 

4  92001687439 
£  61498 


1  N 13404638 

2  E185638371 

3  6160346246 

4  01852854752 
£  35620 


WEAPON  A€  :  1  N01472663 

2  E 184644276 

3  1038565275 

4  07452865375 

5  15832 


WEAPON  A 5 


90 


WEAPON 


WEAPON 


WEAPON 


WEAPON 


WEAPON 


WEAPON 


DOUBLE-SPACED  DISPLAY  FOBKAT 


B1  :  1 

N17694625 

2 

E629575107 

3 

4610375676 

4 

10817381561 

5 

19565 

B2  :  1 

N56872630 

2 

E022957264 

"X 

u 

1295725462 

4 

07362756314 

28158 

B3  :  1 

N  16238375 

2 

E175340833 

** 

7453298136 

4 

08164829154 

FZ 

66264 

B4  :  1 

N66952753 

2 

El 06385672 

3 

1253903647 

4 

01758265081 

5 

21548 

B5  :  1 

N 17694625 

2 

E629575 107 

■x 

O 

4610375678 

4 

10617381561 

C. 

19565 

B6  :  1 

N  15867495 

2 

E973687525 

*2 

1210934753 

4 

53272361071 

fc* 

w 

40912 
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INVERSE-VIDEO  DISPLAY  FORMAT 


WEAPON  B7  :  1  N77896354 

2  E990253146 

3  5421937106 

4  64866351041 

5  86357 


WEAPON  B8  :  1  N57486001 

2  E19665662S 

3  1037561937 

4  18401746218 

5  51073 


WEAPON  Cl  :  1  N01472863 

2  E 184644278 

3  1038565275 

4  07452865375 

5  15832 


WEAPGN  C2  :  1  N 17724497 

2  E82855254? 

3  1850724572 

4  76169473575 

5  29637 


WEAPON  C3  :  1  N 52849654 

2  El0945274y 

3  1075382657 

4  62956272651 

5  20756 


WEAPON  04  :  1  N56194015 

2  E852808461 

3  9108265726 

4  51072656254 

5  77669 


APPENEIX  B 
EXPERIMENTAL  EATA 

TABLE  VIII 

SUBJECT  ERRORS  -  TARGET  LOAEING  TASK 
(UB) 


SUB.# 

i  NIC 

!  NEC 

MCE 

!  NMC 

i  NCE 

I 

1 

TNE  |% 

CORRECT 

ENTR . 

1 

!  0 

!  y 

y 

!  2 

!  1351 

l 

J 

1381  ! 

*y7.e 

1 

1 

2 

!  0 

!  44 

0 

I  2 

i  1316 

1 

1 

1448  1 

£90.9 

1 

1 

3 

!  e 

i  5 

6 

!  l 

!  1354 

1 

1 

1378  | 

%ye.2 

! 

1 

4 

!  46 

I 

51 

!  48 

!  i30y 

1 

1 

1375  | 

%91.9 

I 

l 

P 

1  2 

1  c 

s 

i  2 

!  1352 

1 

I 

1444  | 

£93.  e 

1 

1 

6 

t 

1  ^ 

!  7 

10 

!  3 

!  1350 

1 

1 

1376  j 

£97.  y 

1 

1 

7 

!  0 

!  a 

6 

!  0 

!  1354 

1 

f 

1371  ! 

%y6.e 

1 

1 

6 

i  43 

!  £ 

45 

1  45 

!  1315 

1 

1 

1366  | 

%y3.2 

1 

1 

y 

!  2 

!  12 

12 

!  27 

!  1348 

1 

J 

1399  ! 

£94 . 5 

i 

1 

10 

!  5 

I  ^ 

10 

!  2 

!  1350 

1 

t 

1366  ! 

£97.4 

1 

1 

n 

!  2 

!  l 

1 

I  0 

!  1359 

1 

t 

1362  ! 

£99.8 

1 

1 

12 

!  47 

!  K 

62 

!  yo 

i  1298 

1 

l 

1374  ! 

%ee.7 

J 

1 

13 

!  2 

!  4 

4 

!  0 

!  1356 

\ 

1 

1368  ! 

%yy  .1 

1 

1 

14 

!  2 

!  ** 

P 

!  0 

!  1355 

1 

1 

1394  I 

£97.2 

1 

1 

15 

!  0 

!  22 

22 

i  0 

I  1328 

1 

1 

1428  | 

£93.0 

1 

} 

ie 

!  26 

!  17 

53 

!  46 

1  1307 

1 

I 

1390  ! 

%yi.0 

1 

\ 

17 

i  2 

!  l 

1 

!  0 

!  1359 

1 

1 

1262  | 

£96 .8 

1 

1 

ie 

j  1 

i  5 

7 

!  2 

1  1  ’XC.T 

1 

1 

1381  ! 

%96 . 0 

l 

l 

iy 

!  iy 

!  fc 

27 

1  17 

!  1333 

l 

1 

1369 

%94.6 

1 

20 

!  2 

!  5 

8 

!  2 

!  1252 

/ 

1 

1383  ! 

£97.8 

1 

1 

LEGSNE: 


NIC:  Nurrber  Incorrect  Characters 
NEC:  Number  Entries  Corrected 
NICE:  Nurrber  cf  Incorrect  Entries  =  NIC  +  NEC 
NMC:  Nurrber  Missing  Characters  (sarre  as  NME) 
NCE:  Nunber  Correct  Entries 
TNS:  Total  Number  Errors 
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TABLE  IX 


SUBJECT  ERRORS:  TARGET  LOADING  TASK 


(VOICE) 


SUB.# 

|  NIC 

!  NEC 

!  NICE 

!  NMC 

1  NCE 

1 

i 

TNE  \% 

CORRECT 

ENTR. 

1 

!  1 

i  4 

!  39 

i  0 

i  1221 

1 

1 

1368  ! 

£99.0 

1 

1 

2 

1  2 

!  19 

!  6e 

1  7 

i  1200 

1 

\ 

1412  ! 

%91 . 6 

1 

1 

3 

|  1 

!  17 

!  74 

!  0 

!  1286 

1 

1 

1413  ! 

%yi  .0 

» 

1 

4 

I  4 

1  7 

i  49 

!  2 

i  1311 

1 

1 

1298  ! 

£93.6 

t 

1 

5 

!  3 

!  21 

!  72 

1  'Z 

\ 

i  1287 

1 

t 

1442  ! 

£89. e 

» 

i 

e 

!  8 

!  6 

!  47 

I  7 

i  1313 

1 

1 

1392  ! 

£93.9 

1 

l 

7 

1  T 

1 

i  29 

!  60 

S  0 

!  1310 

J 

1 

1420  ! 

£92.3 

l 

l 

8 

j  1 

!  10 

!  22 

!  45 

!  1349 

1 

1 

1408  j 

£92.8 

1 

\ 

y 

!  0 

!  17 

!  ei 

1  2 

i  1279 

1 

J 

1430  ! 

£89.3 

1 

10 

!  4 

1  82 

!  61 

i  0 

!  1309 

1 

1 

1440  | 

£y0 .9 

i 

1 

*11 

!  l 

i  81 

1  81 

1  0 

!  1279 

( 

1 

1418  ! 

£9e.2 

1 

i 

12 

!  0 

!  18 

!  44 

!  0 

!  1316 

1 

1 

1399  | 

£y4.i 

1 

» 

*12 

!  6 

[  S 

!  41 

!  45 

!  1319 

1 

1 

1378  ! 

£92.7 

1 

1 

14 

!  l 

i  5 

1  22 

!  e 

|  1328 

1 

! 

1292  ! 

£96.1 

i 

1 

*l£ 

!  7 

!  43 

!  89 

!  10 

!  1271 

1 

l 

1449  ! 

£87.1 

1 

l 

16 

!  y 

!  17 

!  108 

i  e 

,  1252 

l 

1 

1444  | 

£86.7 

1 

1 

*17 

!  3 

!  27 

i  118 

1  e 

!  1242 

1 

1 

1436  | 

£86.5 

! 

i 

18 

1  K 

!  ie 

i  41 

!  10 

!  1319 

1 

l 

1400  ! 

£93.5 

( 

1 

*19 

1  1 

!  y 

1  c 

1  Uw 

|  0 

!  1225 

\ 

I 

1391  ! 

£95.2 

1 

1 

20 

!  14 

!  38 

!  lie 

!  y 

i  1250 

I 

1 

1449  ! 

£85.7 

( 

1 

LEGEND: 


NIC:  Number  Incorrect  Characters 
NEC:  Number  Entries  Corrected 
NICE:  Nurber  of  Incorrect  Entries  =  NIC  +  NEC 
NMC:  Nurber  Missing  Characters  (sane  as  NME) 
NCE:  Number  Correct  Entries 
TNE :  Total  Number  Errors 
*  :  Previous  Experience  with  Voice  Input 


(The  voice  portion  of  NICE  also  includes  the  number 
of  voice  unit  ncnrecognitions  and  missrecognitions) 


TABLE  X 


SUBJECT  ERRORS :  TARGET  INFORMATION  VERIFICATION  TASK 


(SINGLE-SPACED  FORMAT) 


SUB.# 

NIC 

NEC  ! 

NICE 

NMC 

i  NCE 

1 

1 

TNE  ! 

%  CORRECT 

ENTR. 

1 

& 

w> 

0  ! 

e. 

1 

i  125 

1 

1 

130  ! 

%95.4 

1 

1 

z 

0 

0  ! 

0 

0 

!  130 

1 

l 

130  ! 

%100.0 

1 

1 

2 

0 

z  ! 

0 

1 

!  130 

1 

1 

132  ! 

%9? .  7 

1 

1 

4 

0 

n  t 

V  | 

0 

0 

|  130 

1 

1 

130  ! 

%100 .0 

1 

1 

4 

l  ! 

c 

0 

!  117 

1 

1 

130  ! 

%90 . 0 

1 

1 

6 

0 

e  ! 

I 

1 

j  129 

1 

1 

130  ! 

%98.5 

1 

1 

7 

0 

0  ! 

0 

0 

|  130 

1 

I 

132  j 

%98 . 5 

1 

t 

6 

1 

0  ! 

1 

0 

!  129 

1 

I 

130  ! 

%92 .3 

1 

1 

9 

0 

0  ! 

0 

0 

!  130 

1 

1 

I3e  ! 

%10e.0 

1 

1 

10 

0 

l  i 

1 

0 

!  129 

1 

1 

132  ! 

%97.7 

1 

1 

11 

0 

e  ! 

6 

1 

!  124 

1 

1 

145  | 

%84 .9 

1 

1 

12 

0 

0  ! 

0 

0 

j  130 

1 

1 

130  ! 

X100.B 

1 

1 

13 

0 

l  ! 

1 

0 

!  I2y 

1 

1 

132  ! 

%9?  .7 

t 

1 

14 

4 

l  ! 

5 

0 

i  125 

1 

\ 

132  | 

%S4.7 

1 

1 

15 

0 

3  ! 

3 

0 

j  127 

\ 

1 

I5e  ! 

%84.6 

1 

1 

ie 

0 

l  ! 

1 

1 

j  I2y 

I 

1 

138  ! 

%92.8 

1 

1 

17 

2 

2  ! 

4 

0 

j  126 

1 

\ 

132  ! 

«95 . 5 

1 

1 

16 

0 

0  i 

0 

0 

j  130 

1 

1 

130  ! 

X100.0 

1 

1 

ly 

0 

0  ! 

0 

0 

I  120 

1 

) 

132  ! 

%98.5 

1 

1 

20 

1 

0  ! 

1 

1 

i  129 

1 

1 

130  i 

%98 . 5 

1 

1 

LEGEND:  NIC: 

NEC : 
NICE: 
NMC : 
NCE : 
TNE: 


Number  Incorrect  Characters 
Number  Entries  Corrected. 

Number  cl'  Incorrect  Entries  =  NIC  ♦  NEC 
Number  Missing  Characters  (saire  as  NME) 
Nunber  Correct  Entries 
Total  Number  Errors 


95 


TABLE  XI 


SUBJECT  ERBORS:  TARGET  INFORMATION  VERIFICATION  TASK 
(DOUBLE-SPACED  FORMAT) 


SUB  .# 

NIC 

NEC  ! 

NICE 

NMC 

!  NCE 

!  TNE  ! 

%  CORRECT 

ENTR. 

1 

0 

4  i 

4 

1 

!  128 

!  141  ! 

ZS0.1 

1 

1 

2 

0 

0  ! 

0 

2 

!  132 

!  136  ! 

X97.1 

1 

1 

% 

o 

1 

£  ! 

6 

0 

i  126 

!  141  ! 

%89.4 

1 

f 

4 

5 

0  i 

5 

0 

1  127 

1  132  1 

%96.2 

1 

1 

b 

0 

0  ! 

0 

0 

!  132 

!  132  ! 

Z100.0 

1 

1 

6 

l  ! 

3 

0 

1  129 

!  134  ! 

%96.3 

t 

1 

7 

0 

l  ! 

1 

0 

i  131 

1  134  ! 

%97.8 

1 

1 

e 

l 

0  ! 

1 

0 

|  131 

!  134  ! 

%96 .3 

1 

1 

9 

l 

3  ! 

4 

0 

i  126 

!  136  ! 

%92 .8 

J 

1 

le 

l 

l 

2 

1 

i  130 

!  134  ! 

%96.3 

1 

1 

n 

0 

l  ! 

1 

1 

!  131 

!  134  ! 

%97.0 

1 

1 

ik 

0 

3  i 

3 

0 

!  129 

1  134  ! 

%96 . 3 

1 

1 

13 

l 

l  ! 

2 

1 

!  130 

!  138  ! 

%93 . 5 

1 

1 

14 

3 

l  ! 

4 

1 

!  126 

!  144  ! 

%66 . 3 

1 

1 

15 

0 

l  ! 

1 

1 

!  131 

I  137  i 

X94.9 

l 

1 

16 

1 

4  ! 

5 

2 

!  127 

!  140  ! 

%69 . 4 

1 

1 

17 

0 

3  ! 

** 

1 

1  129 

!  147  1 

%87 .2 

1 

< 

16 

1 

*1  » 

w  | 

4 

0 

i  128 

!  137  ! 

Z93.4 

1 

1 

19 

1 

2  ! 

3 

0 

!  129 

!  136  ! 

%94 . 9 

1 

I 

20 

0 

3  ! 

3 

0 

!  129 

!  136  ! 

Z93.5 

u 

il 

ii 

il 

LEGEND : 


NIC:  Number  Incorrect  Characters 
NEC:  Number  Entries  Corrected 
NICE:  Number  of  Incorrect  Entries  =  NIC  +  NEC 
MMC:  Number  Missing  Characters  (same  as  NME) 
NCE:  Number  Correct  Entries 
TNE:  Total  Number  Errors 


TABLE  XII 


SUBJECT  ERRORS:  TARGET  INFORMATION  VERIFICATION  TASK 


(INVERSE-VIDEO  SORMAT) 


SUB.# 

!  NIC 

NEC 

MCE 

J  NCE 

■ 

TNE  j 

%  CORRECT 

ENTR. 

1 

i  0 

0 

0 

0 

i  115 

I 

1 

115  ! 

%100  .0 

» 

f 

2 

!  0 

0 

0 

0 

!  115 

( 

1 

121  ! 

%95 . 0 

1 

1 

3 

!  0 

0 

0 

0 

!  115 

1 

1 

115  ! 

2100.0 

1 

1 

4 

!  0 

0 

0 

3 

!  115 

l 

1 

115  | 

297.5 

1 

1 

5 

!  v 

0 

7 

0 

i  108 

1 

1 

115  | 

29  3.9 

1 

1 

e 

!  0 

0 

0 

2 

!  lit 

! 

1 

115  ! 

%96 .3 

1 

1 

7 

!  0 

1 

1 

0 

i  114 

1 

1 

125  | 

%91.2 

1 

) 

e 

!  0 

e 

0 

0 

i  115 

1 

1 

115  ! 

%100.0 

1 

< 

y 

!  0 

i 

0 

0 

!  114 

t 

i 

117  ! 

1 

l 

10 

1  0 

2 

2 

0 

J  113 

l 

f 

119  ! 

295.0 

I 

1 

li 

!  18 

0 

18 

0 

i  97 

\ 

1 

115  : 

284.3 

1 

1 

12 

!  5 

2 

7 

0 

!  108 

1 

l 

115  | 

%93.9 

1 

l 

13 

!  0 

7 

7 

0 

1  108 

1 

1 

121  J 

289.3 

1 

1 

14 

!  3 

1 

4 

1 

i  HI 

1 

1 

127  j 

266 . 7 

1 

1 

15 

!  0 

0 

0 

0 

!  lit 

i 

t 

118  ! 

297.5 

I 

1 

It 

!  9 

O 

12 

1 

I  103 

1 

1 

124  ! 

262 . 4 

1 

1 

17 

|  2 

0 

2 

0 

!  112 

1 

l 

115  j 

298 . 3 

< 

l 

18 

!  0 

0 

0 

0 

!  lit 

1 

1 

115  ! 

2100.0 

i 

\ 

19 

!  l 

2 

3 

0 

J  112 

1 

1 

120  ! 

293.3 

t 

l 

20 

i  l 

2 

1 

0 

I  114 

\ 

1 

115  ! 

299.1 

1 

1 

LEGEND: 


NIC: 

NEC: 

NICE: 

NMC: 

NCE: 

TNE: 


Number  Incorrect  Characters 
Number  Entries  Corrected 
Number  of  Incorrect  Entries  =  N 
Number  Missing  Characters  (same 
Number  Correct  Entries 
Total  Number  Errors 


IC  +  NEC 
as  NME) 


TABLE  XIII 


ERRORS*  LURING  VOICE  ENTRY 


SUB.# 

1 

J 

TARGET  SETS 

TOTAL 

— 

H* 

1 

cn 

e-10 

11-15 

16-20 

1 

!  6  ! 

6 

16 

10 

38 

1 

1 

2 

I  17  ! 

19 

23 

9 

68 

1 

1 

3 

!  14  ! 

26 

21 

11 

72 

1 

1 

4 

!  7  ! 

12 

13 

13 

45 

1 

1 

a 

1  19  j 

18 

16 

15 

70 

1 

1 

6 

i  9  | 

11 

8 

9 

37 

I 

1 

7 

!  12  ! 

11 

6 

6 

9 

1 

\ 

8 

!  7  | 

9 

6 

7 

29 

1 

» 

9 

!  13  1 

18 

24 

26 

81 

1 

1 

10 

!  le  ! 

7 

10 

13 

46 

1 

1 

11 

j  12  ! 

18 

It 

14 

59 

1 

1 

12 

!  n  ! 

9 

6 

9 

35 

1 

l 

13 

!  12  ! 

7 

w 

11 

7 

33 

1 

1 

14 

!  4  ! 

4 

2 

7 

17 

1 

1 

15 

1  13  ! 

6 

7 

13 

39 

1 

1 

16 

!  £  ! 

29 

28 

17 

60 

1 

J 

17 

I  17  J 

17 

26 

26 

88 

1 

1 

18 

1  7  1 

1  1 

2 

6 

9 

20 

1 

1 

19 

!  e  ! 

9 

7 

3 

25 

1 

1 

20 

!  10  ! 

10 

fr. 

23 

48 

J 

1 

mean : 

110.7  | 

12.5 

12.7 

13.0 

sd : 

4.6  ! 

7  .6 

7.6 

6.3 

*  Ncnreccgnlt ions  and  Mlsreccgnitions 


TABLE  XIV 


NCNRECOGNITIONS  Of  UTTERANCES  BY  ENTRY 


ENTRY 


TOTAL  NONRECCGN ITI  ON'S 


0  ! 

92 

1  ! 

49 

2  ! 

86 

3  ! 

31 

4  ! 

1Z? 

£>  1 

%J  1 

82 

6  ! 

44 

7  1 

32 

6  ! 

40 

9  ! 

24 

control ! 

32 

modify  ! 

19 

enter  ! 

69 

yes  1 

4 

alpha  ' 

6 

bravo  ! 

16 

Charlie ! 

0 

left  ! 

1? 

right  j 

32 

TABLE  XV 

NONRECOGNITIONS  Oi  UTTERANCES  BY  SUBJECT 


SUB . : ! TOTAL  NONRECOGNITIONS! 

1  ! 

34 

! 

1 

2  ! 

39 

1 

1 

3  ! 

e  ^ 

V  W 

1 

1 

4  j 

36 

1 

1 

5 

46 

1 

1 

6  ! 

23 

1 

1 

?  I 

16 

1 

1 

8  ! 

16 

I 

1 

9  ! 

63 

1 

1 

10  ! 

20 

1 

1 

ii  ! 

53 

1 

1 

12  ! 

23 

1 

1 

13  ! 

27 

1 

l 

14  ! 

15 

f 

1 

15  I 

39 

1 

1 

16  ! 

81 

1 

1 

17  i 

60 

1 

1 

18  ! 

20 

1 

I 

19  ! 

25 

1 

1 

20  I 

56 

1 

! 
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